Feat/multi tree forest - #114
Merged
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
The pool ran a single depth-20 Merkle tree: 1,048,576 leaves, ever. The tree is append-only and spending a note doesn't free its leaf, so at ~500k private transfers
MerkleTreeFullwould have killed the network permanently.MERKLE_PROOF_SCALING_PLAN.md§5 explicitly deferred this decision; this PR is that decision, followingMULTI_TREE_FOREST_PLAN.md(Railgun's model: seal a full tree, roll over to a new one).What
Rollover. When an insert fills the tree it seals it in the same call: the final root goes to permanent storage, the frontier resets, and the next insert starts a fresh tree. The global u32 leaf index never resets (
tree_id = leaf_index / MaxLeavesPerTree), which is what keeps event shapes, indexer chunking and wallet scan cursors untouched.Sealed roots are permanent — this is the load-bearing part. The historic ring holds 100 roots and evicts on every insert. A naive port would have left a sealed tree's final root in there: 100 inserts into the new tree and every unspent note in the old one becomes permanently unspendable. That's not downtime, it's lost funds. So
SealedTreeRoots/SealedRootIndexare separate permanent maps (bounded by tree count, max 4096), andis_known_rootchecks the ring OR the sealed set. The transfer/unshield/validate_unsignedcall sites are unchanged — they already went through that one function.No circuit, VK, extrinsic or ABI changes. Anchoring was already root-only (the chain never asks "which tree?"), and
nullifier = Poseidon2(commitment, sk)carries no tree information, so cross-tree nullifier collisions aren't possible. The commitment-dedup and nullifier sets stay global across the forest — a duplicate commitment in a later tree would alias its nullifier and let one spend nullify both notes.Config
MaxLeavesPerTree(production 2^20, mocks 8 so rollover is testable).integrity_testenforces power-of-two ≤ 2^MAX_TREE_DEPTH: clients derivetree_idfrom this constant, so changing it on a live chain would re-map every note.Runtime API v2 adds
get_forest_infoandget_root_for_leaf. The privacy RPC resolves each leaf's anchoring root through it, so a note in a sealed tree gets its permanent root instead of the live one — which also removes the root-drift retry problem for those notes. Response gains an additivetree_id.STORAGE_VERSION2 with a version-only migration (sealed maps start empty). Forest invariants merged into the existingtry_state: active root always known, one sealed root per completed tree, sealed maps bijective.Testing
Pallet suite 286 green (with
--features try-runtime), precompile 65, RPC 35.Rollover-specific unit tests: seal fires with the right event order (
MerkleRootUpdatedcarrying the final root, thenTreeSealed); the anti-freeze regression — a sealed root survivesMaxHistoricRoots + Nlater inserts and its notes still verify; boundary straddling (leaves 7 and 8 land in different trees); duplicate commitment rejected across trees; multi-rollover with every leaf provable against its own tree's root;try_stateholds across a seal.Verified against a running node (dev runtime with an 8-leaf cap, scripts in
ts-tests/): 20 shields seal trees 0 and 1 with distinct roots; all 20 leaves resolve to their own tree's anchoring root; lookup-by-commitment agrees with lookup-by-index on both root and path; sealed roots still correct after the later trees filled.Security review
One real finding, fixed: the RPC proof handlers fell back to the active root when a leaf's anchoring root couldn't be resolved — that would have served a sealed-tree path with the wrong anchor. They now error explicitly. Fail-closed, not fail-wrong.
Verified as non-exploitable: proving against the empty root requires a Poseidon preimage; the
size % cap == 0boundary resolves to the sealed root; level-0 siblings can't cross a tree boundary (local ^ 1stays inside); duplicate empty roots in the ring are covered by the existing eviction guard; the seal's ~4 uncharged writes are a once-per-million event, not a DoS vector.Deploy notes
spec_version4 → 5.transaction_versionunchanged.get_root_for_leaf). Upgrade the runtime before or together with the binaries.